The process of categorising an event by its deviation from some predetermined pattern or theory is termed anomaly detection. The process of compliance monitoring for anomaly detection (CMAD) involves a primary monitoring system comparing some predetermined conditions of acceptance with the actual data or event. These primary monitoring systems typically use templates, cases, threshold levels (filters) or checklists, separately or in combination. If any variance is detected by the primary monitoring system, an exception report or alert is produced, identifying the variance. In a simple environment this identification of the variance fulfils the conditions of necessary and sufficient evidence and thus determines an instance of non-compliance. In a more complex environment it may be only an indicator of possible non-compliance. In the latter case further evidence will be required to substantiate the hypothesis of non-compliance. The function of a CMAD system is therefore two-fold, namely identifying a variance, and producing and accumulating (if required) supporting evidence.
In a complex environment, CMAD decision making is ex post, more involved and may require multiple steps. The event monitoring and decision making is in a domain where the initial monitoring uses a priori thresholds broader than in a simple environment, i.e., more granular. This initial monitoring produces exceptions that identify suspected non-compliant events (SNCEs). Once these exceptions have been produced, it is then the task of the decision maker to substantiate true positive exceptions. True positives are those exceptions that the decision maker has determined are indeed anomalous and where the evidence supports this assertion. To obtain this supporting evidence the decision maker uses the results of the initial monitoring as well as important information, related to the event, and characterised by its interpretive nature, requiring judgmental expertise. The decision maker may also need to identify, categorise and discard any false positive exceptions. These are exceptions that have signalled suspected events that require further scrutiny, and are subsequently rejected by the decision maker, for various reasons. On the other hand, false negatives are events for which the current monitoring facilities do not generate an exception, and allow possible suspect events to slip through the CMAD sieve. If the initial monitoring threshold limits are stringent enough, it can be argued that the marginal false negatives could be subsumed and later considered. Nevertheless, this would not necessarily reduce the occurrence of true false negatives as their characteristics may not be known.
CMAD has been employed in the data intensive capital market, in which products are traded through different types of orders by market participants, who follow market rules and comply with regulatory structures. Market participants evaluate products and analyse news to determine when to place orders. Regulatory bodies monitor news and market activity to determine when participants are not in compliance with market rules. Conduct which is in breach of market rules and exchange regulations include instances of insider trading and various forms of market manipulation. An example of a prior art CMAD system in the capital market (CMADcm) is the current surveillance operation at the Australian Stock Exchange (ASX), which uses an analytical model based on the statistical matching approach to CMADcm. It combines computer-based decision support systems to analyse market events with communication software, text retrieval and graphics. The system, surveillance of market activity (SOMA), includes related sub-systems such as real-time monitoring of market events, news display, market replay, and alerts history. SOMA originated from the New York Stock Exchange's (NYSE) STOCK WATCH system and was modified for the Australian context. SOMA primarily uses statistical methods (means, variances, moving averages, days since last traded, etc.) to identify SNCEs.
Problems that are found with the prior art analytical compliance monitoring models include the following:
Difficulties arise because in general, (1) details of the SNCE source agent may not be known and must be discovered or inferred from the data; (2) the definition of ‘unusual pattern of behaviou’ is subjective and possibly changes with every analysis and over time; and (3) the quantity of the data in an analysis is overwhelming. Other problems encountered with analytical models include (4) incomplete model theories—models often contain incomplete theories as well as incomplete data; (5) incomplete model inputs—even the best models occasionally produce decisions much worse than a human analyst would, because they do not include some important factors; (6) incomplete model outputs the analys's risk preference in dealing with uncertain outcomes might differ from that of the model. Conversely, the analysts role is trivialised if the model makes all the decisions; and, (7) incomplete explanations—models provide precision at the expense of intuition and common sense.
These analytical, predictive and compliance models are often rejected by the decision-makers. Consequently, to compensate for these limitations, some analysts “tune” the results by making heuristic adjustments to the analytical model. This tuning produces a model forecast that is consistent with intuitive expectations, and maintains the detail and structure of the analytical model. However, tuned forecasts can easily be misused. Alternatively, a cognitive model of an analyst, implemented as an expert system, might perform better at predictive tasks than an analytical model. However, probability based cognitive models fail in domains where there is too much reliance on judgment. In these domains, judgments are dynamic and their representation is difficult to quantify and verify.